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ABSTRACT 

This  report  addresses  the  use  of  instructional  media  in  teaching  executive  management 
education.  While  it  is  clear  that  using  technology  to  support  distance  learning  can  be  cost 
effective,  this  research  shows  that  benefits  may  be  lost  for  learners  and  instructors  in  highly 
interactive  learning  environments.  This  research  proposes  four  learning  outcomes,  which  are 
considered  to  be  consistent  with  graduate  education,  and  examines  them  in  the  context  of  the 
instructional  techniques  required  to  support  them.   Media  selection  guidelines  were  then 
developed  from  the  analysis.   Data  were  collected  from  the  developers  of  32  modules  of 
graduate-level,  management  education  that  are  used  for  Navy  medical  officers.  Module 
outcomes,  instructional  techniques,  and  the  expected  stability  of  the  learning  content  were 
analyzed  and  the  media  selection  guidelines  were  used  to  match  each  module  with 
videoteletraining,  digitized  video  disc,  or  traditional  classroom  instruction. 


BACKGROUND 


Introduction 


Congressional  legislation  mandates  that  prospective  Commanding  Officers  and 
Executive  Officers  of  Military  Treatment  facilities  (MTF)  must  demonstrate  managerial 
competence  before  they  can  assume  these  job  roles.  To  provide  the  education  necessary  to 
demonstrate  that  competence,  the  Navy  Bureau  of  Medicine  and  Surgery  (BUMED)  in 
partnership  with  the  Naval  Postgraduate  School,  Systems  Management  Department,  designed 
an  Executive  Management  Education  (EME)  program  tailored  to  the  needs  of  Navy  health 
care  professionals.  The  curriculum  contains  over  460  hours  of  graduate-level  management 
education  ranging  from  understanding  budgeting  to  managing  change. 

The  current  EME  program  (that  is,  modules  that  have  been  developed  and  were 
available  for  analysis)  consists  of  38  modules,  ranging  in  length  from  4  to  24  hours,  which  are 
delivered  at  the  MTFs.  This  on-site  module  delivery  plan  was  developed  because  it  is  not 
feasible  for  health  care  personnel  to  leave  their  jobs  to  enter  a  resident  program.  However,  this 
delivery  method  may  not  provide  the  modules  fast  enough  to  the  officers  who  need  them. 
Furthermore,  because  officers  frequently  transfer  commands,  EME  program  administrators 
will  have  difficulty  ensuring  that  modules  that  an  officer  or  a  small  cadre  of  officers  need  are 
available  at  their  command. 

To  solve  this  problem,  EME  administrators  are  examining  the  feasibility  and 


appropriateness  of  using  various  types  of  instructional  media  such  as  videoteleducation  (VTE) 
systems  and  digitized  video  discs  (DVD)  to  deliver  EME  modules.  To  provide  EME 
administrators  with  the  data  and  analysis  they  need  to  formulate  a  workable  instructional 
delivery  plan  that  does  not  compromise  learning,  NPS  Systems  Management  researchers  are 
conducting  studies  that  assess  the  lessons  learned  from  VTE  research  and  applications;  that 
empirically  test  for  differences  in  learning,  interaction,  and  student  satisfaction  between  EME 
modules  delivered  via  VTE  and  traditional  face-to-face  instruction;  and  that  offer  a  theoretical 
model  describing  two  distinct  ways  that  administrators  and  teachers  think  about  education  that 
influence  VTE  system  design  and  pedagogy  (Suchan  &  Crawford,  1995).   This  report  broadens 
the  scope  of  the  inquiry;  it  addresses  to  what  extent  and  in  what  situations  VTE  and  DVD  can 
be  used  to  support  the  EME  Program. 


Purpose  of  the  Research 


The  objective  of  the  research  is  to  develop  a  heuristic— a  set  of  guidelines—that  BUMED 
educators  can  use  to  help  select  instructional  media  to  support  course  modules  in  executive 
education.   These  guidelines  are  reflected  in  a  matrix  that  links  specific  BUMED  EME 
modules  with  appropriate  instructional  media.   The  matrix  is  derived  from  five  primary 
factors,  their  relationships  with  each  other,  and  the  actions  as  a  result  of  those  factors. 

1 .  Analyze  the  needs  of  the  learner  and  the  organization  to  determine  the  appropriate 
instructional  content. 

2.  Match  instructional  content  with  learning  outcomes,  that  is,  the  performance 
expected  as  a  result  of  the  learning  experience. 

3.  Select  instructional  techniques  that  support  specific  learning  outcomes.  Instructional 
techniques  vary  with  respect  to  levels  of  interactivity . 
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4.  Select  instructional  media  that  will  support  the  levels  of  interactivity  represented  by 
the  selected  instructional  techniques. 

5.  Modify  the  media  selection  decision  based  on  the  expected  stability  of  the 
instructional  content  and  cost  considerations. 

This  report  examines  each  of  these  factors  in  detail  and  shows  the  interrelated 
influences  that  make  instructional  media  choice  a  complex  strategic  activity.  This  complexity  is 
partly  caused  by  differences  in  control  of  these  factors:  learning  outcomes  may  be  determined 
by  administrators,  instructors,  learners,  or  combinations  of  these  groups;  instructional 
techniques  are  generally  determined  by  the  instructor;  and  media  technology  availability  and 
use  is  often  determined  by  administrators  and  technology  experts.  These  differences  in  control 
make  it  challenging  to  align  learning  needs,  learning  outcomes,  and  instructional  techniques 
with  instructional  media  choice. 

The  integration  of  these  factors  results  in  the  matrix  for  instructional  media  selection. 
The  matrix,  which  relies  heavily  on  widely  accepted  concepts  of  adult  learning  and  media 
choice  based  on  media  richness,  applies  as  well  to  other  executive  education  programs. 

Before  discussing  the  media  selction  factors,  we  provide  a  brief  overview  of 
instructional  media  choice  as  a  strategic  activity.  This  view  that  instructional  media  should  be 
chosen  as  a  result  of  carefully  assessing  a  number  of  important  variables  is  the  central  premise 
of  this  report. 


Media  Choice  as  a  Strategic  Activity 


Media  have  unique  and  defining  characteristics.  These  characteristics,  which  will  be 
discussed  in  detail  in  a  later  section,  determine  the  type  of  communication  the  media  can 
support.  In  many  communication  situations,  managers  and  leaders  must  strategically  choose  the 
right  media  or  combination  of  media  to  achieve  a  desired  outcome  (Daft,  R.L.,  Lengel,  R.H., 
&  Trevino,  L.  K.,  1987).  Inappropriate  media  choice  can  sabatoge  a  manager's  best  efforts  to 
achieve  that  outcome.  In  fact,  strategic  media  choice  is  an  important  ingredient  of  leadership. 
Savvy  leaders  constantly  make  shrewd  media  choice  decisions  to  achieve  specific  outcomes 
such  as  extending  their  presence  through  the  organization  or  changing  organizational  thinking 
and  behavioral  patterns  by  using  a  variety  of  media  to  communicate  new  vision,  values,  and 
strategic  goals  (Trevino,  L.K.,  Daft,  R.L.,  &  Lengel,  R.H.,  1990). 

Until  recently,  training  and  executive  development  administrators  and  instructors  have 
not  been  faced  with  significant,  large  scale  instructional  media  choice  decisions.  Instructors 
taught  face-to-face  in  classrooms,  and  administrators  merely  needed  to  ensure  that  there  was 
the  right  mix  of  students,  classrooms,  audio-visual  equipment,  and  instructors. 

That  has  changed.  Advances  in  instructional  technologies  have  created  for  both 
administrators  and  instructors  new,  complicated,  and  in  some  cases  expensive  media  choice 
decisions.  Detennining  whether  BUMED  executive  management  education  modules  can  be 
offered  via  new  technologies,  and,  if  they  can,  deciding  which  modules  lend  themselves  to 
these  media  and  determining  the  right  mix  of  media  has  become  an  important  strategic  media 
choice  decision. 

Instructional  media  must  be  able  to  support  a  variety  of  instructional  techniques  such  as 
lecture,  question  and  answer,  discussion,  dialogue,  simulations,  and  role  plays.  Some  of  these 
approaches  enable  learners  to  take  individual  and  collective  responsibility  for  their  own  and 
others'  learning.  Media  that  enable  learners  to  help  direct  the  content  of  their  own  learning  can 
improve  the  quality  of  learning  and  also  the  satisfaction  of  learners  (Sherry,  1996). 

6 


Because  these  new  media  are  costly  and  provide  a  radically  different  medium  for 
learning,  the  decision  stakes  are  high.  Inappropriate  media  choice  decisions  can  waste  millions 
of  dollars  in  underutilized  "high-tech"  equipment,  undermine  learning,  and  sour  learners  about 
the  value  of  the  learning. 


FACTORS  INFLUENCING  MEDIA  SELECTION 

This  section  addresses  five  factors  that  are  proposed  as  the  primary  elements  to 
consider  in  media  selection. 

Organizational  and  Adult  Learning  Needs 

Organizational  needs  will,  of  course,  always  be  the  starting  point  for  any  new  program 
to  train  or  educate  individuals  in  the  organization.  In  the  case  of  executive  education  for  Naval 
medical  personnel,  the  impetus  for  the  new  program  began  with  the  Congressional  legislation 
mentioned  at  the  start  of  this  report.   Additionally,  fundamental  structural  changes  in  the 
military  health  care  delivery  system  have  challenged  providers  to  meet  their  stakeholders' 
needs.  These  changes  require  new  knowledge,  strategies,  and  skills  to  balance  effective 
utilization  of  health  care  assets  with  high-quality  care.  Even  in  the  early  1980s,  Neufeld  and 
Barrows  (1984)  argued  that, 

As  a  result  of  the  knowledge  explosion  and  technological  revolution  in 
the  last  half  of  this  century,  the  rate  of  obsolescence  of  human  beings  has  been 
increasing  exponentially.   In  few  areas  of  life  has  this  obsolescence  been  more 
pronounced  than  in  the  medical  profession.   (Neufeld  and  Barrows,  1984,  p. 
207) 


Without  doubt,  the  rate  of  clinical  and  administrative  knowledge  obsolescence  has 
increased.  Consequently,  lifelong  clinical  and  managerial  learning  through  continuing 
education  is  a  critical  aspect  of  the  careers  of  all  medical  professionals.   Indeed,  concepts  like 
"learning  organizations,"    "continuous  improvement,"  "lifelong  learning,"  and  "knowledge 
creating  organizations"  have  been  used  to  characterize  an  essential  feature  that  all  organizations 
must  have  if  they  are  to  remain  competitive  in  the  next  20  years.  Senge  argues  that  "as  the 
world  becomes  more  interconnected  and  business  becomes  more  complex  and  dynamic,  work 
must  become  more  'learningful.'"  (Senge,  1090,  p. 4)  Indeed,  BUMED  members  have 
experienced  the  organizational  jolt  that  interconnectedness  can  cause  as  a  result  of  the 
movement  to  TRICARE  and  the  Lead  Agency  concept. 

Continuous  learning  implies  rapid  change  in  knowledge,  strategies,  and  abilities. 
Consequently,  the  structure  and  content  of  executive  medical  education  must  be  flexible 
enough  to  meet  officers'  current  and  future  career  development  needs. 

As  a  result  of  the  Congressional  legislation  and  rapid  changes  in  the  medical  field, 
BUMED  created  organizational  strategies  to  respond  to  these  changes.  One  strategy  was  a 
needs  analysis  to  identify  the  specific  instructional  content  required  for  the  EME  program  and 
the  learning  needs  and  expectations  of  the  intended  audience.  In  the  case  of  the  EME  program, 
the  audience  consists  of  highly  sophisticated  learners.   The  physicians,  nurses,  administrators, 
and  and  other  health  professions  represented  have  high  expectations  for  the  relevance  of 
learning  to  their  jobs,  how  learning  should  occur,  and  their  roles  in  the  process. 

Research  on  adult  learning  demonstrates  that  adults  generally  learn  more  deeply  and 
permanently  with  instructional  approaches  that  give  them  the  opportunity  to  use  their  own 
initiative  as  compared  to  traditional  teacher-oriented  classroom  approaches  (Savery,  J.R.  & 
Duffy,  T.M.,  1995).  These  differences  in  learning  result  because  the  adult  learner  is  self- 
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directing  and  accustomed  to  taking  responsibility  for  all  aspects  of  his/her  life.  Medical 
community  professionals  possess  these  characteristics;  generally  they  have  high  growth  needs, 
eagerly  take  responsibility  for  their  learning,  and  desire  autonomy.  Knowles  (1984)  points  out 
that  this  desire  for  self-direction  presents  an  interesting  problem  for  instructors: 

For  even  though  adults  may  be  totally  self-directing  in  every  other  aspect  of 
their  lives—as  workers,  spouses,  parents,  citizens,  leisure-time  users-the  minute 
they  walk  into  a  situation  labeled  'education,'  'training,'  or  any  of  their 
synonyms,  they  hark  back  to  their  conditioning  in  school,  assume  a  role  of 
dependency,  and  demand  to  be  taught.  (Knowles,  1984,  p.  9). 

The  educator  who  uses  for  all  learning  situations  a  traditional,  lecture-based,  teacher-oriented 
classroom  approach  in  which  the  adult  learner  has  no  control,  may  create  a  conflict  because  of 
learners'  needs  for  control.   If  the  educator  does  not  resolve  this  conflict,  the  adult's  energy 
may  be  diverted  from  learning. 

This  adult  learning  framework  also  accounts  for  the  greater  quantity  and  quality  of 
experience  that  adults  bring  to  the  learning  situation  compared  to  younger  learners  such  as 
undergraduate  students.  This  job-related  experience  and  know-how  can  only  be  shared  if 
instructors  construct  environments  that  create  opportunities  for  learners  to  interact  easily  with 
each  other.  Knowles  (1984)  argues  that  for  many  kinds  of  learning,  adult  students,  not 
instructors,  are  the  richest  learning  resources  for  one  another.   In  fact,  both  Senge  (1990)  and 
Nonaka  and  Takeuchi  (1995)  claim  that  significant  organizational  learning  occurs  only  when 
individuals  have  the  opportunity  to  place  their  individual  insights,  experiences,  and  best 
practices  within  the  larger  pool  of  group  or  collective  experience  and  test  the  organizational 
applicability  of  those  insights  and  practices.  Through  instructor-facilitated  discussion, 
dialogue,  and  even  role  plays,  the  individual  and  group  not  only  can  explore  complex  issues 
from  multiple  perspectives  and  gain  insights  unavailable  from  individual  self-reflection  and 
analysis  but  also  forge  new,  shared  collective  understanding  that  can  reshape  organizational 
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thought  and  behavior. 

Capitalizing  on  the  learning  experiences  of  executives  through  discussion,  dialogue,  and 
role  plays  requires  an  instructional  environment  that  enables  significant  verbal  and  non-verbal 
interactivity  and  allows  for  rich  feedback.  Furthermore,  these  student-centered  modes  of 
learning  require  instructors  to  function  as  process  facilitators.  Consequently,  if  a  new 
instructional  technology  is  used  to  reach  these  executives,  that  technology  must  be  capable  of 
supporting  not  only  these  interactive,  communication  rich  modes  of  learning  but  also  the 
facilitation  role  (and  the  communication  requirements  that  define  that  role)  instructors  must 
play. 

To  summarize,  the  research  in  adult  learning  shows  that  adults,  particularly 
professionals,  thrive  in  communication  rich,  highly  interactive  learning  environments 
fundamentally  different  from  the  more  structured,  instructor-centered  environment  that 
students  with  fewer  life  and  work-related  experiences  learn  in.  The  research  shows  that  adults 
learn  most  effectively  in  environments  in  which  they  can  exercise  initiative  and  control; 
experience  interactive,  collective  learning  through  shared  experiences  with  and  feedback  from 
their  peers;  and  respond  to  learning  materials  that  are  contextualized  for  their  professions. 
Indeed,  adults,  particularly  those  in  management  development  programs,  enter  a  learning 
situation  motivated  to  perform  more  skillfully  a  task,  to  solve  more  effectively  a  problem,  to 
make  a  better  quality  decision,  or  to  learn  to  think  differently  about  organizational  situations. 
In  short,  they  want  to  take  immediate  action  as  a  result  of  what  they  have  learned    (Edwards, 
1992).  Consequently,  learning  experiences  must  be  thoroughly  grounded  in  the  context  of 
specific  professions,  which  require  constantly  updated  contextualized  learning  materials. 
Furthermore,  learners  must  be  provided  the  opportunity  to  test  and  better  understand  their 
insights  from  their  experiences  through  discussion  and  other  more  learner-centered  teaching 
techniques. 


10 


Although  we  have  made  a  strong  case  for  the  creation  of  instructional  environments  that 
enable  professionals  to  be  responsible  for  and  direct  their  learning,  two  fundamental  questions 
arise:  to  what  extent  do  adults  need  to  be  involved  in  their  learning  and  does  this  degree  of 
involvement  apply  to  all  learning  situations?  In  the  next  section  we  discuss  the  second  and 
third  media  selection  factors.  This  explication  shows  that  distinct  learning  outcomes  exist 
(Gagne  and  Briggs,  1979)  and  to  achieve  each  outcome,  instructors  must  use  different 
instructional  techniques  and  strategies  and  the  right  combination  of  instructional  media  to 
support  those  techniques. 


Learning  Outcomes  and  Instructional  Techniques 


Educators  and  learners  have  goals,  often  called  learning  outcomes,  that  are  achieved 
through  various  communication  methods  commonly  called  instructional  techniques  and 
strategies  or  pedagogies.  Different  instructional  strategies  and  techniques  strategically  chosen 
and  combined  create  unique  learning  environments  where  students  may  achieve  the  desired 
learning  outcomes.  Consequently,  executive  education  should  be  conducted  in  different 
instructional  environments  depending  on  the  requirements  of  the  learning  outcome  desired. 
Four,  distinct  learning  outcomes  and  the  instructional  techniques  that  define  their  pedagogical 
environments  are  described.1 

Know  and  supply  information.   This  category  includes  knowledge  of  facts,  principles, 
concepts,  procedures,  and  rules.  Examples  from  this  category  are  knowledge  of  contracting 
regulations,  basic  principles  of  accounting,  rules  and  regulations  of  civilian  employment,  and 
the  Program  Planning  and  Budgeting  System.   This  knowledge  can  serve  as  a  building  block 
for  more  advanced  outcomes.  Also,  this  type  of  knowledge  can  have  a  pragmatic  function: 


1  We  have  adapted  Gagne  and  Briggs'  (1979)  learning  outcomes  to  a  hypothetical  framework  that  we  believe 
to  be  consistent  with  the  outcomes  of  graduate  education. 
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mid-level  executives  can  supply  executives  or  decision  makers  with  the  knowledge  to 
understand  a  concept,  procedure,  or  rule  before  taking  appropriate  action.  For  example,  an 
MTF's  director  of  administration  could  inform  the  CO  about  the  financial  guarantees  that  are 
part  of  contracts  let  out  to  civilian  health-care  providers. 

When  learning  basic  concepts  in  an  unfamiliar  discipline,  students  are  dependent  on  the 
teacher's  knowledge  and  ability  to  communicate  clearly  that  information.  Generally,  students 
learn  highly  structured,  self-contained  information  that  they  need  to  memorize,  or  to  know 
where  the  information  is  located  and  how  they  can  quickly  access  it.  Consequently,  knowledge 
of  these  basic  concepts  and  structured  procedures  can  be  "transferred"  from  instructor  to 
student.  This  knowledge  transfer  can  be  accomplished  by  lecture,  question  and  answer,  and 
limited  discussion.  These  instructional  techniques  require  minimal  interaction,  consisting 
primarily  of  periodic  feedback  to  individuals  in  an  question/answer  format  to  ensure  correct 
comprehension  of  concepts. 

Tapping  learner  experience  is  not  required  to  attain  this  learning  outcome;  however, 
this  may  create  conflict  in  highly  self-directed  professionals.  Consequently,  it  is  incumbent 
upon  the  educator  to  remind  their  self-directed  adult  learners  that  there  are  some  situations  that 
require  lecture-oriented,  instructor-centered  learning.  Students'  engagement  with  the  material 
is  internal;  learners  need  to  enter  such  situations  in  a  searching,  probing  frame  of  mind  (an 
internal  self-directedness)  that  will  enable  them  to  connect  new  concepts  with  prior  knowledge 
to  create  a  richer  pattern  of  associations  (Edwards,  1992). 

Apply  information  within  structured  situations.  This  outcome  requires  the  learner 
not  only  to  "know"  but  also  to  "do."  The  student  learns  what  knowledge,  principles,  concepts, 
rules,  and  procedures  to  apply  to  structured  problem-solving  and  decision  making  situations. 
For  example,  learners  would  be  able  to  apply  the  correct  contract  regulation  to  a  procurement 
request;  to  know  how  to  perform  the  Total  Quality  Leadership  plan,  do,  check,  act  process;  or 
to  select  the  appropriate  accounting  principle  for  a  financial  management  decision.  Although 
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there  may  some  degree  of  uncertainty  in  these  situations,  this  learning  outcome  deals  primarily 
with  correct  and  incorrect  processes,  problem  solutions,  and  decisions.  In  other  words,  there  is 
great  likelihood  that  knowledgeable  managers  would  implement  the  same  (or  similar)  process, 
solution,  or  decision.   Since  this  category  involves  the  application  of  knowledge  to  a  wide 
range  of  work-related  situations,  a  moderate  level  of  intructor-learner  interactivity  is  required. 
For  example,  instructors  must  discuss  with  students  a  variety  of  examples  and  non-examples  of 
situations  where  the  knowledge  can  be  applied.  These  examples  must  be  current  and  grounded 
in  the  learners'  professional  context  so  that  they  are  meaningful  to  them.  Students  then  can  be 
asked  to  extend  the  knowledge  and  procedure  applications  by  providing  additional  examples 
and  by  describing,  based  on  their  professional  experiences  and  observations,  the  consequences 
or  errors  and  the  reasons  why  errors  are  made. 

Discussion  of  how  the  knowledge,  rules,  or  procedures  can  be  applied  to  the  example 
situations  is  instructor  directed.  Instructors  demonstrate  these  applications  and  then  ask 
students  to  apply  them  to  given  situations;  instructors  then  give  feedback  through  open-ended 
questions  (coaching)  or  direct  statements  (evaluative  feedback)  to  correct  student  errors. 
Consequently,  instructors  trying  to  develop  this  student  learning  outcome  must  not  only  be  able 
to  lecture  well  but  also  must  be  skilled  at  giving  and  eliciting  feedback.   These  instructional 
techniques  require  not  only  subject-matter  expertise  but  also  the  ability  to  ask  pointed 
questions,  "read"  students  to  determine  who  is  having  difficulty  learning,  and  know  when  it  is 
time  to  move  on  to  a  new  application. 

Exercise  judgment  in  the  face  of  uncertainty.  This  learning  outcome  develops  in 
executives  the  ability  to  exercise  judgment  when  faced  with  complex,  messy,  and  sometimes 
career-threatening  problems  of  strategy,  change  management,  program  management,  ethics, 
and  leadership.  This  outcome  defines  judgment  broadly  to  include  problem  analysis, 
assessment  of  internal  and  external  stakeholder  needs,  determination  of  system-wide 
implications  of  implementing  a  problem  solution,  and  a  variety  of  other  activities  that  require 
executives  to  analyze  and  synthesize  both  "hard"  and  "soft"  information.   Unlike  the  previous 
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category,  this  outcome  specifies  performance  in  the  absence  of  firm  rules  and  guidelines  for 
applying  learned  knowledge  and  skills. 

This  outcome  defines  much  of  what  senior  executives  do:    "Judgment  despite 
uncertainty  is  essentially  the  art  of  management. "  (Harvard  Business  School,  1994)  Executives 
must  be  constantly  adding  information  to  their  decision-making  templates  or  schema—a  highly 
personalized  "bank"  of  interconnected,  organizational  knowledge  used  to  make  difficult 
decisions—so  as  to  make  them  more  robust.  Furthermore,  they  must  be  flexible  enough  to  use  a 
variety  of  "lenses,"  depending  on  the  context,  to  "see"  problems  and  situations  from  the 
multiple  perspectives  of  their  stakeholders.  When  a  number  of  management  tools  are  available, 
they  must  know  which  one  has  a  high  probability  of  producing  the  desired  outcome.  To  do 
this,  they  must  reconcile  disparate  evidence  and  learn  to  make  tradeoffs,  to  prioritize,  to  decide 
what  data  are  most  important  to  the  decision. 

This  outcome  requires  the  executive  to  understand  and  use  a  systems  perspective  of 
organizations.  The  executive  has  a  high-level  view  of  the  organization  with  a  clear 
understanding  of  its  many  components  and  the  ways  in  which  they  connect  and  interact.   The 
effective  executive  understands  that  changing  one  part  of  the  organization  will  have  ripple- 
effects  throughout  the  organization.  Thus,  executives  must  consider  a  wide  variety  of 
variables  before  making  a  decision.  Furthermore,  though  an  executive  may  understand  the 
probable  effects  of  a  decision,  he  or  she  will  be  unable  to  predict  the  specific  outcome  that 
will  result  from  that  decision.  Examples  of  these  type  of  problems  include  ethics-based 
decision  making,  determining  when  it  is  risky  versus  wise  to  use  bottom-line  writing, 
manipulating  reward  systems  versus  changing  structure  to  enhance  motivation,  or  using  one 
strategy  versus  another  to  manage  large-scale  change. 

The  instructional  techniques  required  within  this  learning  domain  are  clearly  learner- 
centered.  Instructors  would  be  wise  not  to  lecture  executives  on  how  to  make  an  ethical 
decision.  Although  ethical  frameworks  can  be  taught  as  a  basic  concept  within  the  "Know  and 

14 


Supply  Information"  category,  ultimately  executives  make  ethics-based  decisions  under  very 
equivocal  circumstances  that  require  them  to  draw  on  their  deep  understanding  of  organizations 
based  on  both  education  and  experience.  Instructors  must  tap  these  executive  wellsprings  of 
experience  so  that  executives  can  collectively  develop  their  understanding  of  the  myriad 
situational  factors  that  influence  an  ethical  decision.  To  do  so,  highly  interactive,  and  feedback 
intensive  instructional  strategies  are  required.  For  example,  the  instructor  can  use  the  Socratic 
method  to  help  executives  clarify  and  refine  ideas,  role  plays  that  require  executives  to  attend 
to  complex  interpersonal  dynamics,  open-ended  questions  that  enable  executives  to  share  and 
through  the  articulation  of  their  experiences  better  understand  them,  and  case  studies  that 
enable  executives  to  formulate  their  own  and  understand  others'  strategies  for  dealing  with  a 
complex  organizational  situation. 

Within  this  learning  environment,  instructors  function  as  coaches  and  facilitators;  their 
role  is  to  surface  executive  experience  through  appropriate  prompts  (questions,  cases,  role 
plays),  orchestrate  the  interactions  and  feedback  so  that  executives  will  see  trends  and  patterns, 
and  create  a  climate  of  trust  so  that  executives  will  not  hesitate  to  share  failures  as  well  as 
successes. 

The  value  of,  and  expectation  for,  bringing  the  executive's  experience  to  the  classroom 
was  illustrated  in  a  week-long  Navy  Medical  Department  Executive  seminar  that  previewed 
segments  of  the  EME  program.  Medical  executives  provided  feedback  about  the  seminar  and 
the  EME  program  in  highly  interactive  focus  groups  conducted  at  the  end  of  the  week.   Group 
members  said  that  a  primary  advantage  of  the  program  is  the  opportunity  to  bring  together 
groups  of  colleagues  from  different  medical  areas  who  rarely  have  the  opportunity  to  share 
their  insights  and  thus  learn  from  each  other.  These  BUMED  executives  saw  this  type  of 
individual  and  collective  learning  through  discussion,  debate,  and  feedback  as  necessary  to 
prepare  BUMED  officers  for  executive-level  positions. 
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Understand  and  change  habits  of  mind:  Paradigm  breaking.  This  last  learning 
outcome  attempts  to  make  executives  self-conscious  about  the  strategies  they  customarily  use  to 
assess  complex  problems  and  to  be  aware  of  the  assumptions  that  are  at  the  foundation  of  these 
strategies.  Sometimes  called  double  loop  (Argyris  &  Schon,  1978)  or  generative  learning 
(Schon,  1983),  the  results  of  this  outcome  may  not  immediately  be  a  better  repertoire  of 
strategies,  but  executives  who  are  more  aware  of  the  way  they  typically  attack  problems,  the 
sources-professional,  educational,  and  personal— of  the  problem-solving  strategies  they 
customarily  use,  and  the  strengths  and  limitations  of  their  approaches.  This  awareness  or 
heightened  self-consciousness  may  eventually  lead  to  new  patterns  of  thinking  that  can  result  in 
breakthrough  interpretations  of  organizational  situations  and  novel  approaches  to  organizational 
problem  diagnosis. 

For  example,  many  executives,  particularly  those  from  a  hard-sciences  background,  see 
communication  as  the  "transfer"  of  meaning  and  believe  that  language  "contains"  meaning.  An 
executive  who  sees  communication  as  information  transfer  will  devise  communication 
strategies  that  both  reflect  and  reinforce  that  view.  Consequently,  if  the  task  is  to  communicate 
a  new  mission,  vision,  and  values  throughout  the  organization,  he  or  she  may  use  a  wide  array 
of  written  channels— written  documents,  postings,  e-mail  messages,  electronic  tack  boards— to 
transfer  meaning  to  organizational  members.  That  strategy,  driven  by  the  information  transfer 
view  of  communication,  may  fail.  And  because  the  executive  is  locked  into  this  information- 
transfer  model  of  communication,  he  or  she  may  attribute  the  failure  to  others  or  to  poor 
implementation  of  the  strategy  rather  than  flawed  assumptions  about  communication.  In  short, 
the  executive  is  caught  in  a  dysfunctional  communication  decision  making  loop,  and  does  not 
realize  it. 

To  break  that  loop,  a  shift  of  mind  or  a  different  mental  model  about  communication  is 
necessary.  That  shift  of  mind  begins  with  the  executive's  recognition  of  the  model  of 
communication  being  used,  an  understanding  of  the  limitations  of  that  model,  and  an 
appreciation  for  the  way  that  model  shapes  one's  thinking  and  action  about  communication 

16 


issues.  Only  after  this  realization  has  taken  place,  can  the  executive  entertain  different  habits  of 
mind  about  communication~for  example,  meaning  exists  in  the  audience,  communication  is 
generation  of  shared  understanding,  communication  is  process  requiring  redundancy  to  clarify 
new  managerial  approaches,  miscommunication  is  the  norm-and  develop  strategies  based  on 
this  different  understanding. 

This  discussion  of  potentially  dysfunctional  habits  of  thinking  about  communication 
issues  can  be  extended  to  leadership,  motivation,  change,  decision  making,  reward  and 
incentive  systems,  and  a  number  of  other  complex  managerial  situations. 

Generating  this  learning  outcome  is  difficult.  In  fact,  only  the  executives  themselves 
can  create  it.  Through  prompts  such  as  cases,  scenarios,  simulations,  and  role  plays, 
instructors  create  conditions  that  enable  executives  "to  put  on  display"  their  thinking  processes, 
their  habits  of  mind.  While  these  processes  are  occurring,  the  instructor  must  slow  down  the 
executives' s  thinking  processes  through  pointed  questions  and  the  inputs  of  other  executives 
(collective  feedback)  so  that  the  learner  can  become  aware  of  the  characteristics  of  his  or  her 
mental  model,  the  inherent  assumptions  behind  that  model,  the  blind  spots  the  model  creates, 
and  the  ways  in  which  the  mental  model  steers  problem  interpretation  and  strategic  action. 

Creating  alternative  habits  of  mind  and  the  strategic  thinking  and  action  that  result  from 
these  new  habits  is  the  collective  responsibility  of  the  executives.  Because  of  their 
organizational  knowledge,  executives  should  be  more  insightful  than  the  instructor  in 
generating  innovative,  "outside-the  box"  approaches  to  knotty  organizational  issues.  For  this 
learning  outcome  to  be  successful,  executives  must  "suspend"  their  assumption,  literally  hold 
them  at  arm's  length  in  front  of  the  group.   Executives  must  regard  each  other  as  colleagues  in 
learning  and,  clearly,  the  instructor  must  be  able  to  create  a  high  level  of  trust. 

As  in  the  previous  category,  the  instructor  functions  as  a  learning  process  facilitator; 
however,  to  help  the  group  engage  in  dialogue  that  may  result  in  novel  interpretation,  a 
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heightened  facilitation,  which  is  a  kind  of  artistry,  is  required.   This  requires  a  heightened 
sensitivity  to  individual  and  group  non-verbal  signals  that  enables  the  group  to  discover  new 
interpretations  rather  than  to  slip  in  the  mode  of  stating  and  defending  positions.  Unlike  the 
previous  category  in  which  the  learning  process  involves  adding  knowledge  and  skills  to  the 
executive's  repertoire,  this  category  requires  learners  to  fundamentally  change  the  way  they 
think  about  organizations.   Consequently,  there  may  well  be  resistance  to  these  changes.    As 
Kantor  (1992)  has  noted,  change  to  new  behaviors  can  be  interpreted  in  a  framework  that 
suggests  that  previous  behaviors  were  wrong,  which  can  serve  as  a  powerful  deterrent  to 
change.   The  instructor/facilitator  in  this  situation  must  be  skilled  at  managing  these 
perceptions. 


Media  Characteristics 


The  previous  section  described  four  distinct  categories  of  learning  outcomes  and  the 
instructional  techniques  and  strategies  needed  to  reach  those  outcomes.  This  section  uses  media 
richness  theory  to  analyze  the  different  capabilities  of  media  that  support  various  forms  of 
communication  needed  to  deploy  instructional  strategies  and  techniques  so  as  to  attain  various 
learning  outcomes. 

Media  choice  researchers  have  demonstrated  the  need  to  match  communication  task 
requirements  with  media  that  have  capabilities  to  meet  those  requirements,  These  media 
capabilities  can  be  classified  on  a  continuum  from  "lean"  to  "rich."  Relatively  lean  media  have 
sufficient  information  carrying  capacity  to  fulfill  structured,  routine  communication  task 
requirements  such  as  information  exchange  or  knowledge  transfer.  With  more  unstructured  and 
ambiguous  tasks,  such  as  strategic  planning  in  a  dynamic  environment,  there  is  a  greater  need 
for  rich  media  to  enable  people  to  generate  multiple  interpretations  of  available  information, 
explore  different  solution  scenarios,  and  come  to  sufficient  agreement  about  the  meaning  of  the 
problem,  the  interpretation  of  the  data,  and  the  steps  to  take  (Lengel,  R.H.,  &  Daft,  R.L., 
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1988). 


In  this  context,  one  of  the  key  requirements  of  educators  is  to  choose  the  instructional 
media  that  have  the  communication  capabilities  to  support  the  instructional  strategies  required 
to  achieve  the  desired  learning  outcomes. 

Daft,  Lengel,  and  Trevino  (1987)  graft  media  on  a  lean  to  richness  continuum.  We 
have  modified  their  model  to  include  the  emerging  instructional  media  such  as  VTE  that  are 
not  included  in  their  original  model.   There  are  three  interrelated  factors  that  detennine  the 
relative  leanness  and  richness  of  media:  the  capability  of  the  medium  to  support  various  levels 
of  verbal  and  non-verbal  interactivity,  the  type  and  timeliness  of  feedback  a  medium  can 
provide,  and  the  relative  degree  of  perceived  interpersonal  presence  that  the  media  can  enable. 

Interactivity  is  a  medium's  ability  for  senders  and  receivers  to  notice  and  respond  to 
each  others'  communication  cues.  There  are  verbal  and  non-verbal  components  to  interaction: 
language,  speech  rate,  voice  pitch  and  intonation,  eye  contact,  proxemics  (use  of  space), 
gestures,  and  body  language  in  general.  Different  media  support  to  varying  degrees  one  or 
more  of  these  components,  thus  enabling  different  types  and  amounts  of  interaction.  Feedback 
is  subcategory  of  interaction.  We  define  feedback  as  a  specific  type  of  interaction  that  has  as  its 
purpose  the  development  of  individual,  group,  or  organizational  learning.  Finally, 
interpersonal  presence  is  the  media's  ability  to  communicate  the  personal  feelings  and  emotions 
that  accompany  many  face-to-face  communication  interactions  (Daft,  R.L.,  Lengel,  R.H.,  & 
Trevino,  L.  K.  1987). 

How  well  and  easily  a  medium  supports  a  particular  component  of  interactivity 
influences  the  extent  to  which  the  component  is  used.  For  example,  if  an  instructor  using  a 
two-way  video  and  audio  videoteleducation  system  finds  it  too  difficult  or  cumbersome  to  "cut" 
from  a  long  shot  of  the  class  to  a  medium  close-up  of  a  student  asking  a  question,  that 
technological  limitation  will  steer  the  instructor  into  relying  on  long  shots.  That  reliance  will 
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affect  the  interactivity  between  instructor  and  learner.  The  instructor  will  be  forced  to  focus 
merely  on  the  language  of  the  question  and  not  the  student's  facial  expression,  the  tilt  of  the 
head,  and  the  position  of  the  body.  The  limited  interactivity  because  of  the  need  to  focus 
merely  on  language  will  influence  the  content  and  tone  of  the  instructor's  response,  the 
possibility  of  follow-up  questions  and  responses,  and  even  the  possibility  that  the  questions 
could  open  up  a  rich  discussion  among  other  learners.  In  essence,  limited  interactivity  could 
also  limit  the  amount  and  type  of  learning  enhancing  feedback  between  the  instructor  and 
learners. 

Rich  media  both  make  available  and  support  through  ease  of  use  the  full  range  of 
interactivity  components.  Provided  below  are  several  illustrations  of  the  media  richness  criteria 
applied  to  specific  media. 

On  the  media  richness  continuum,  written  communication  (non-e-mail)  is  considered  a 
lean  medium.  This  medium  provides  limited  interactivity.  There  are  no  multiple  cues  from 
sender  and  receiver  to  clarify  or  reinforce  the  meaning  of  the  message.  And  because  of  the  lack 
of  multiple  cues,  the  sender  and  the  receiver  have  very  limited  interpersonal  presence.   That  is, 
the  medium  enables  limited  awareness  of  the  other  person  and  the  opportunity  to  develop  an 
interpersonal  relationship  between  the  sender  and  receiver.  Finally,  the  medium  provides 
delayed  feedback,  if  any  at  all. 

One  reason  correspondence  courses,  a  form  of  "distance  learning,"  have  high  drop  out 
rates  and  low  learner  satisfaction  is  precisely  because  learners  get  delayed  feedback  about  the 
quality  of  their  learning.  This  delay  may  make  it  difficult  to  connect  feedback  about  how  well 
the  task  was  performed  with  the  receiver's  awareness  of  how  he/she  performed  the  task; 
consequently,  the  impact  of  the  feedback  on  learning  and  learner  motivation  may  be  minimal. 
Furthermore,  because  the  medium  is  capable  of  supporting  limited  instructor/student 
interactivity  due  to  the  lack  of  multiple  cues,  the  feedback  may  seem  cryptic,  incomplete,  or 
lacking  in  thoroughness  to  the  learner.  Finally,  the  lack  of  interpersonal  presence  between 
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instructor  and  learner  caused  by  limited  interactivity  may  dampen  learner  motivation  to 
continue  the  correspondence  program.  Finally,  adult  learners  may  perceive  they  have  no 
control  over  their  learning.  This  perception  may  also  dampen  motivation. 

Another  lean  medium  is  digitized  video  disc  (DVD),  which  provides  the  student  with 
computer-based  instruction  with  access  to  massive  amounts  of  information.   Single  learners 
engage  with  a  multimedia  system  to  progress  through  multiple  sequences  of  learning.  These 
systems  enable  learners  to  branch  out  to  additional  information  for  elaboration  of  a  concept  and 
for  sequences  of  remediation  or  acceleration.   The  automated  feedback  provided  by  DVD  is 
timely  but  generic.  It  cannot  be  designed  to  parallel  the  abilities  and  experiences  of  all  of  the 
students  in  the  program.   Thus,  it  may  or  may  not  be  sufficient  to  meet  individual  learning 
needs.   By  definition,  interactivity  is  limited;  DVD  cannot  "read"  cues  that  learners  provide. 
Also,  interpersonal  presence  is  lacking. 

A  one-way  compressed  video  (instructor  movement  is  somewhat  balky),  two-way  audio 
videoteleconferencing  system  is  a  moderately  rich  medium.  Because  the  instructor  cannot  see 
the  student,  non-verbal  cues  are  missing,  thus  significantly  limiting  interactivity.  Furthermore, 
factors  such  as  monitor  size  and  quality;  room  set-up;  the  number,  type,  and  quality  of 
microphones;  and  the  variety  of  camera  shots  (visual  interest)  affect  the  type,  amount,  and 
quality  of  interactivity  between  instructor  and  learner.  This  limited  interactivity  makes  it 
difficult  for  the  instructor  to  determine  the  type  and  amount  of  feedback  to  meet  learner  needs. 
As  a  result,  feedback  content,  though  rapid  and  timely,  may  be  limited  to  learner  questions  and 
clarifying  answers  that  may  in  turn  limit  learning  to  knowledge  of  information  and  the  correct 
application  of  programmed  procedures.   Finally,  interpersonal  presence  is  lacking  because 
students  are  merely  voices  from  a  distant  location.  As  a  result,  trust,  openness,  and  instructor- 
learner  cohesion  may  be  very  difficult  to  develop. 

Full-motion,  two-way  video  and  two-way  audio  systems  that  have  multiple, 
strategically  placed  cameras  with  sophisticated  camera  "cutting"  features  that  enable  learners 
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and  instructors  to  see  medium  closeups  of  each  other  almost  immediately  after  they  start 
speaking  is  considered  a  moderately  rich  medium.  Feedback  is  timely,  though  voice  and 
movement  may  be  slightly  out-of  phase.  Two-way  video  enables  multiple  cues  thus  enabling 
increased  interactivity.  Also,  careful  shot  selection  can  make  clear  body  language  and  function 
to  some  degree  as  a  substitute  for  proxemics  (the  manipulation  of  physical  distance  between 
instructor  and  learner  to  create  different  interaction  effects). 

Two-way  video  and  audio  systems  are  classified  as  only  moderately  rich  for  the 
following  reasons.  Even  with  careful  shot  selection,  interactivity  is  limited  because  many  non- 
verbal signals  may  be  washed  out  or  impossible  to  see.  This  varies  with  the  sophistication  of 
the  technology  available.   Furthermore,  when  the  camera  zooms  to  a  close-up  of  a  learner 
speaking,  the  instructor  is  unable  to  see  the  rest  of  the  group  and  simultaneously  attend  to  their 
non-verbals  (e.g.,  who  wants  to  speak  next,  is  confused,  is  "getting  it,"  is  bored,  etc.)  These 
limits  to  interactivity  affect  the  dynamics  of  open-ended  discussions,  case  analyses,  and  role 
plays  where  attention  to  subtle  cues  and  split-second  feedback  timing  is  necessary  for  these 
conversation-rich  techniques  to  be  successful.  Additionally,  interpersonal  presence  is 
dampened  by  the  medium:  instructors  and  learners  remain  two-dimensional  visual  icons.  This 
may  make  it  difficult  to  develop  the  trust  necessary  to  engage  in  personal  idea  testing. 

These  descriptions  of  various  media  illustrate  the  need  for  administrators  and 
instructors  to  strategically  choose  media  so  that  there's  a  match  or  an  alignment  between  the 
communication  capabilities  of  the  media—sufficient  interactivity,  feedback,  and  interpersonal 
presence—and  instructional  techniques  and  learning  outcomes. 


Economic  Considerations  and  Content  Stability 


Once  the  first  four  media  selection  factors  have  been  worked  through,  there  are  some 
practical  issues  that  must  be  considered.    First,  what  are  the  cost  constraints,  and  second,  how 
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stable  is  the  instructional  content? 

Where  there  is  a  choice  of  media  that  will  support  learning  requirements,  it  is 
reasonable  to  select  the  lowest-cost  alternative.   Unfortunately,  media  decisions  are  often 
strongly  influenced  by  cost  without  regard  to  learning  requirements.   For  example,  many 
administrators  would  like  to  link  scores  of  electronic  classrooms  and  offer  all  management 
training  and  education  via  videoteleducation  and  training  systems.  However,  basing  these 
decisions  solely  on  economic  factors  ignores  the  differences  in  learning  outcomes  and  the 
techniques  required  to  support  those  outcomes.   In  the  case  of  management  training  and 
education,  which  may  encompass  all  four  learning  outcomes  and  thus  multiple  instructional 
techniques,  traditional  classroom  instruction  with  scores  of  students  at  remote  locations  would 
not  produce  effective  learning.   Running  the  class  by  the  more  lean  medium  of  VTE  would 
force  instructors  to  use  instructional  techniques  that  would  undermine  the  instructors'  and 
learners'  ability  to  achieve  the  more  complex  learning  outcomes  discussed  earlier.  In  short, 
economic  considerations  would  compromise  attaining  learning  outcomes. 

The  second  consideration  is  the  stability  of  the  learning  content.  As  noted  earlier,  the 
instructional  content  in  the  medical  domain  will  need  to  be  changed  often.   Much  of  what 
medical  professionals  learn  in  executive  education  must  be  offered  through  a  medium  that  can 
be  continuously  and  easily  modified  so  that  materials  are  up-to-date.  Indeed,  media  that  enable 
instructors  to  quickly  and  easily  change  course  content  is  a  key  criterion  of  a  media  choice 
strategy. 

With  these  final  considerations  in  mind,  the  next  section  describes  the  alignment 
between  specific  instructional  media  and  instructional  techniques. 
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MATCHING  MEDIA  WITH  INSTRUCTIONAL  TECHNIQUES 
AND  LEARNING  OUTCOMES 

This  discussion  focuses  on  the  minimum  richness  requirements  a  medium  must  possess 
to  support  the  instructional  techniques  required  to  attain  the  various  learning  outcomes.  Media 
must  be  capable  of  supporting  the  highest  level  outcome  and  the  most  communication  rich 
strategy  the  instructor  is  trying  to  realize. 

Furthermore,  new  media  such  as  DVD  and  VTE  have  more  traditional  media  embedded 
in  them  (e.g.,  text  and  visuals);  consequently,  these  technologies  occupy  a  range  on  the 
richness  scale.  The  location  on  the  range  depends  on  how  effectively  instructors  make  full  use 
of  a  technology's  capabilities. 


Know  and  Supply  Information 


This  learning  outcomes  requires  minimal  instructor/learner  interaction,  and  learners 
generally  require  limited  feedback.  Consequently,  relatively  lean  media  can  be  used  to  attain 
this  outcome.  The  lean  media  requirements  enable  administrators  to  focus  on  cost-saving 
technologies  such  as  video-taped  lectures  or  multi-point  (a  number  of  linked  sites)  VTE  that 
can  reach  a  large  number  of  students.  In  fact,  emerging,  relatively  low-cost  technologies  such 
as  the  Internet  and  the  World  Wide  Web  can  be  piggy-backed  onto  any  one  of  these  previously 
mentioned  media  to  supply  not  only  information  via  Web  pages  but  also  to  provide  student 
feedback  via  e-mail. 

If  the  content  or  procedures  to  be  learned  remain  stable  for  a  long  time  and  the 
instruction  must  reach  large  numbers,  then  professionally  developed,  digitized  video  discs  or 
video-taped  lectures  (VHS  format)  with  high  production  values  delivered  by  a  dynamic  speaker 
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will  enable  students  to  meet  this  learning  outcome. 

The  drawback  of  extensively  using  lean  media  to  train  mature  adult  learners  with  high 
growth  needs  is  decreased  motivation.  As  mentioned  earlier,  seasoned  professionals, 
particularly  executive-level,  health-care  officers,  often  demand  control  over  their  learning  and 
expect  instruction  that  requires  them  to  interact  with  colleagues.  Lean  media  concentrate  the 
locus  of  control  in  the  instructor  or  the  technology  itself. 


Apply  Information  Within  Structured  Situations 


This  outcome  requires  moderately  interactive  instructional  techniques  such  as 
instructor-focused  discussion  and  extended  question/answer  exchanges.  To  support  these 
techniques,  media  located  at  the  mid-level  of  the  richness  scale  are  needed.  Consequently, 
automated  instructional  media  such  as  computer-based  instruction  or  DVD  do  not  provide  the 
interaction  that  can  be  provided  by  an  on-site  instructor. 

VTE  systems  have  the  capability  of  supporting  the  communication  these  instructional 
techniques  require.  In  fact,  one-way  video  and  two-way  audio  VTE  systems,  a  significantly 
less  costly  alternative  to  two-way  video  and  audio  VTE  systems,  are  communication  rich 
enough  to  support  the  instructor/learner  interaction  and  feedback.  Although  the  one-way  video 
makes  it  impossible  for  instructors  to  respond  to  multiple  student  cues  (visual  and  spatial),  the 
structured  nature  of  this  learning  outcome  may  require  only  verbal  interaction  between 
instructor  and  learner. 
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Exercise  Judgment  in  the  Face  of  Uncertainty 


This  outcome  requires  highly  interactive,  feedback-intensive  instructional  techniques 
such  as  discussion,  case  analysis,  and  role  plays.  These  techniques  require  rich  communication 
media.  Obviously,  face-to-face  instruction  is  the  richest  medium.  However,   a  cleverly 
designed  full-motion,  two-way  video  and  two-way  audio  VTE  system  may  be  able  meet  the 
richness  criteria.  This  system  would  require  state-of-the  art  equipment  at  the  originating  and 
remote  sites.  For  example,  Hewlett-Packard  has  constructed  a  television  studio-quality 
originating  site  for  their  product  and  salesperson  video  training  programs.  To  employ  highly 
interactive,  feedback-intensive  instructional  techniques,  instructors  need  to  use  the  equipment 
to  its  maximum  communication  capacity.  This  use  may  require  technical  and  graphics  support. 

Although  a  robust  VTE  system  may  be  able  to  support  the  instructional  strategies  this 
learning  outcome  requires,  creating  the  interactive,  feedback-rich  learning  environment  is 
considerably  more  difficult  via  VTE  than  in  the  traditional  face-to-face  classroom.  The  medium 
acts  as  a  filter  to  the  free-flowing  interaction  that  can  occur  in  face-to-face  communication.  For 
example,  even  the  most  sophisticated  satellite  broadcast  systems  have  a  voice  transmission  time 
lag  that  can  make  conversation  awkward.  This  time  lag  significantly  affects  routines  for 
entering  and  exiting  conversation  that  instructors  and  learners  unconsciously  use. 
Consequently,  the  free-flowing,  dynamic  exchanges  that  characterize  well-run  case  analyses 
and  role  plays  are  difficult,  if  not  impossible,  to  duplicate  in  the  VTE  environment. 
Furthermore,  different  remote  location  microphone  systems—some  of  which  link  an  open  mike 
with  a  medium  close-up  shot  of  the  person  speaking—require  learners  to  time  the  opening  and 
closing  of  their  mikes  so  that  the  instructor  can  hear  the  conversation  and  see  the  person 
speaking.  This  requirement  dampens  spontaneity  and  can  result  in  a  halting,  slowed-down 
discussion. 

Even  in  true-full  motion  (30  frames  per  second),  two-way  video  VTE  systems,  the 
camera  and  the  monitor  do  not  function  like  human  eyes.  This  lack  of  full  physical  fidelity  can 
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make  it  difficult  to  determine  who  is  speaking  at  remote  sites,  particularly  if  the  speaker  is  in 
the  back  of  a  traditionally  designed  classroom.  Also,  this  lack  of  fidelity  makes  it  very  difficult 
for  instructors  to  analyze  non-verbal  cues  to  determine  who  is  confused,  bored,  or  wishes  to 
speak  next. 

Although  these  and  other  factors  dampen  interaction  and  limit  feedback,  we  do  not 
know  to  what  extent  these  differences  affect  learning  or  how  much  they  may  be  mitigated  as  a 
result  of  instructors'  and  learners'  changing  communication  behaviors  as  a  result  of  continuing 
use  of  and  greater  familiarity  with  VTE  systems.  Clearly,  the  limitations  these  factors  create 
require  that  instructors  make  the  effort  to  learn  about  and  use  special,  VTE-specific  interaction 
and  feedback  techniques  for  activities  such  as  case  analysis  and  role  plays.   We  believe  these 
techniques  may  partially  mitigate  the  dampening  effect  VTE  has  on  interaction.  This  mitigation 
would  enable  the  instructor  to  use  adequately  the  teaching  strategies  required  to  achieve  the 
desired  learning  outcome. 

In  short,  VTE  use  causes  a  tradeoff  between  the  amount  of  interaction  as  well  as  the 
quality  of  feedback  and  cost  effective  access  to  learners.  Given  financial  constraints,  the 
tradeoff  may  be  well  worth  it. 


Understand  and  Change  Habits  of  Mind:  Paradigm  Busting 

This  outcome  requires  instructional  techniques  that  are  highly  interactive  and  feedback 
intensive.  Furthermore,  instructor  interpersonal  presence  is  a  key  factor  to  establish  the 
learning  climate  to  achieve  this  outcome.  To  function  as  facilitators,  instructors  must  be 
perceived  as  trustworthy  and  be  able  to  help  establish  that  climate  in  the  learning  environment. 
Consequently,  a  very  rich  medium  is  required.  We  believe  that  only  the  face-to-face  medium  is 
capable  of  meeting  this  requirement  because  of  its  ability  to  be  highly  interactive,  enable  varied 
types  of  feedback,  and  support  the  interpersonal  presence  of  the  instructor  and  learners. 
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To  create  the  conditions  for  this  outcome,  the  instructor  as  facilitator  must  artfully 
create  a  safe  learning  environment  where  executives  can  leave  their  job  roles  and  positions  "at 
the  door. "  Furthermore,  the  facilitator  must  generate  trust  so  that  executives  will  be  less  likely 
revert  to  defensive  routines  that  protect  them  from  the  embarrassment  and  threat  of  "exposing" 
their  own  thinking  and  making  clear  the  assumptions  and  weaknesses  behind  that  thinking. 

Clearly,  the  interpersonal  presence  of  the  facilitator  and  executives  is  essential  to  create 
the  trust  and  non-defensiveness  that  such  forms  of  interaction  and  feedback  require.  Even  the 
most  sophisticated  VTE  systems  cannot  duplicate  the  complex  interplay  of  visual,  spatial, 
tactile,  and  olfactory  cues  that  generate  interpersonal  presence  and  enable  the  facilitator  to 
create  a  safe  learning  environment.  In  the  most  general  sense,  VTE  does  not  allow  a  facilitator 
to  reach  out  and  touch  someone.  The  facilitator  remains  a  two-dimensional,  digitized  visual 
icon  that  exists  only  on  a  monitor. 

In  addition  to  the  difficulty  of  generating  trust,  even  robust  VTE  systems  are  unable  to 
support  the  amount,  type,  and  quality  of  interactivity  this  learning  outcome  requires.  For 
example,  the  VTE  system  at  the  Naval  Hospital  in  San  Diego  requires  users  to  click  on  their 
mike  to  speak  so  that  the  camera  zooms  in  on  the  person  speaking;  this  process  governs  all 
verbal  interaction  at  the  remote  site.  Although  this  zoom  to  a  medium-close-up  enables  the 
remote  instructor  to  better  read  the  speaker's  visual  cues,  the  instructor,  however,  cannot  see 
the  reactions  of  others  to  the  comment.  This  complex  microphone  and  camera  shot  selection 
system  makes  it  impossible  for  learners  to  interact  with  each  other  in  a  spontaneous,  free- 
flowing  manner,  let  alone  for  remote  facilitators  to  generate  dialogue,  keep  it  dynamic,  and 
skillfully  intervene  when  defensive  routines  begin  to  occur. 

In  addition,  these  systems,  as  discussed  in  the  previous  section,  are  unable  to  provide 
the  multiple,  fine-grained  visual  and  auditory  cues  that  are  a  prerequisite  for  quality  debate  and 
discussion.  Consequently,  only  small-group,  face-to-face  seminars  have  the  requisite  amount  of 
communication  medium  richness  to  employ  the  instructional  strategies  needed  to  attain  this 
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learning  outcome. 


LEARNING  OUTCOMES  OF  BUMED  MODULES 

We  surveyed  NPS  BUMED  module  instructors  to  determine  learning  outcomes  of  each 
of  their  modules  (the  modules  are  shown  in  Appendix  A),  the  instructional  techniques  used  to 
achieve  those  outcomes,  and  the  expected  stability  of  the  content  of  their  modules  (the  survey 
is  included  as  Appendix  B).  These  data  were  compiled  in  a  matrix  that  matches  these  factors 
with  the  media  rich  enough  to  help  achieve  the  desired  learning  outcomes.   Table  1  shows  the 
data  collected  from  the  instructors.  Five  modules  are  not  represented  in  the  table  because  the 
instructors  were  not  available  to  provide  the  data.   Table  2  shows  the  match  between  each 
module  and  the  media  that  could  be  used  for  delivery.  The  criteria  used  for  matching  the 
modules  to  media  are  discussed  below. 
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Table  1.  BUMED  Module  learning  outcomes,  learning  techniques, 
and  expected  rate  of  change. 
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Table  2.  Media  Recommendations  for  BUMED  Modules 


TABLE  2 


VTE  CANDIDATES 
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DVD  Criteria 


Stability  of  module  content  was  the  primary  criterion  used  to  determine  which  modules 
might  lend  themselves  to  DVD  application.   As  Table  1  indicates,  module  content  stability  is 
divided  into  five  categories  that  represent  the  amount  of  change  in  content  a  module  would 
undergo  within  three  years.   Module  developers  determined  the  percentage  of  change  in 
module  content  by  checking  the  appropriate  category. 

As  indicated  earlier,  DVD  technology  would  be  cost  effective  for  module  content  that 
could  be  expected  to  remain  relatively  stable  for  an  extended  period.   The  high  cost  of  initial 
DVD  preparation  would  make  modules  in  which  content  changes  quickly  poor  candidates  for 
DVD  application.   Stable  content  is  operationalized  here  as  equal  to  or  less  than  20  percent 
change  in  module  content  within  a  three-year  period. 

Another  criterion  used  to  determine  DVD  module  applicability  was  the  learning 
outcomes  of  the  module  and,  indirectly,  the  pedagogical  techniques  used  to  meet  those 
outcomes.   DVD  provides  learners  with  structured  learning  heuristics  that  are  anticipated  and 
devised  by  subject  matter  experts.   Consequently,  DVD  technology  well  supports  learning 
outcomes  that  reflect  knowledge  acquisition  (e.g.,  knowledge  of  accounting  concepts  and 
contracting  regulations),  structured  processes  (e.g.,  following  the  correct  HAZMAT 
procedures),  and  even  some  complex  decision  processes  where  there  is  sufficient  information 
to  use  a  structured  process  for  making  a  decision  (e.g.,  constructing  a  stakeholder  map). 
These  outcomes  are  reflected  in  the  Know  and  Supply  Information  and  Apply  Information  to 
Structured  Processes  categories. 

In  addition  to  the  content  stability  criterion,  we  also  stipulated  that  at  least  50  percent  of 
a  module's  learning  outcomes  must  reflect  these  two  categories  for  a  module  to  be  a  good 
candidate  for  DVD  application.   As  Table  2  indicates,  eight  modules  meet  these  criteria. 
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VTE  Criteria 


The  amount  of  interactivity  a  module  requires  was  the  primary  criterion  used  to 
determine  VTE  module  applicability.   This  "amount,"  which  varies  from  module  to  module, 
was  implicit  in  the  instructional  techniques  module  developers  use  to  achieve  specific  learning 
outcomes.  Specifically,  module  instructors  who  rely  on  lecture  and  question/answer 
techniques  require  less  interaction  between  themselves  and  students,  and  between  and  among 
students,  than  instructors  who  use  discussion,  cases,  role  plays,  interactive  exercises,  and 
other  forms  of  experiential  learning. 

In  addition,  since  learning  outcomes  desired  should  influence  instructional  techniques, 
we  examined  the  data  to  determine  if  there  was  a  relationship  between  a  module  developer's 
determination  of  learning  outcomes  for  a  module  and  the  instructional  techniques  used  to 
achieve  those  outcomes.   Except  for  modules  numbered  36  and  37,  there  is  a  very  strong 
relationship  in  the  expected  direction.   Modules  focusing  on  Know  and  Supply  Information  and 
Apply  Information  to  Structured  Processes  depend  heavily  on  lecture  and  question/answer 
techniques.   On  the  other  hand,  the  learning  outcomes  of  Exercise  Judgment  and  Change 
Habits  of  mind  are  associated  with  more  interactive  pedagogy. 

The  earlier  discussion  of  VTE  implies  that  using  the  medium  for  specific  learning 
applications  is  not  a  simple  yes/no  choice.  Because  interactivity  is  a  continuum,  we  suggest 
three  categories  of  the  VTE-module  "fit."  These  categories,  as  shown  in  Table  2,  suggest  that 
some  modules  are  excellent  candidates  for  VTE,  others  are  satisfactory,  and  yet  others  are 
high-risk  VTE  candidates. 

Excellent  candidates  for  VTE  are  classified  as  modules  that  use  50  percent  or  more 
lecture  and  question/answer,  and  focus  50  percent  or  more  on  the  Know  and  Supply 
Information  and  Apply  Information  to  Structured  processes  learning  outcomes.  These  modules 
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are  excellent  candidates  because  they  do  not  require  interpersonnaly  complex  interaction  to 
attain  their  learning  outcomes.   Consequently,  the  dampening  effect  that  VTE  systems  have  on 
more  complex  interpersonal  interactions  is  not  an  issue  when  the  interaction  is  highly 
structured  and,  often,  instructor  directed. 

Modules  in  this  category  may  also  allow  for  multipoint  delivery  sites.   In  fact,  a 
worthwhile  research  project  would  be  to  determine  the  number  of  additional  sites  that  could  be 
added  to  a  particular  delivery  before  learner  satisfaction  and  learning  drops  off.   In  addition, 
modules  in  this  category  might  be  offered  via  a  one-way  video  and  two-way  audio  VTE 
system.   This  type  of  system  would  significantly  reduce  the  delivery  costs.  Table  2  shows  that 
15  modules  meet  the  criteria,  including  the  eight  modules  that  are  also  recommended  for 
DVD. 

Satisfactory  candidates  for  VTE  are  classified  as  modules  that  use  50  percent  or  more 
of  the  highly  interactive  instructional  techniques  (everything  other  than  lecture  and 
question/answer),  and  focus  50  percent  or  more  on  the  Exercise  Judgement  and  Change  Habits 
of  Mind  learning  outcomes. 

Modules  that  meet  these  criteria  are  considered  only  satisfactory  candidates  for  VTE 
because—according  to  the  literature  reviewed  and  the  analysis  presented  here—there  are 
potential  difficulties  in  using  VTE  technology  to  support  the  amount  of  interactivity  required 
for  the  instructional  techniques  used  for  these  high-level  learning  outcomes.         Literature 
reviewed  and  interviews  conducted  by  the  present  researchers  (Suchan  and  Crawford,  1995) 
indicate  frustration  on  the  part  of  some  learners  and  instructors  when  VTE  is  used  where 
extensive  interpersonal  interaction  between  and  among  all  participants  is  required.   While  there 
is  insufficient  research  to  determine  the  actual  impact  on  learning  effectiveness,  there  is  some 
risk  of  a  decrement  in  learning  and  learner  satisfaction,  particularly  among  senior  managers,  if 
VTE  is  used  for  less-structured  learning  outcomes.  We  have  recently  completed  data 
collection  on  a  study  that  addresses  whether  there  are  decreases  in  learning  outcomes  and 
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interactivity  between  VTE  and  a  face-to-face  version  of  a  moderately  interactive  BUMED 
module. 

A  factor  that  mitigates  potential  learner  frustration  and  dissatisfaction  is  availability  and 
perceived  access  to  the  education.  Learners  tend  to  be  more  tolerant  of  the  decreased  fidelity  of 
a  VTE  system  and  system  breakdown  if  they  perceive  they  need  the  education  and  would  have 
difficulty  obtaining  it  from  traditional  face-to-face  instruction.   Similarly,  learner  expectations 
of  the  educational  experience—based  on  experience  with  interactive  learning  in  the  traditional 
classroom  format—will  moderate  frustration  with  technology-based  education.   Learners 
experienced  in  face-to-face  interactive  education  are  less  likely  embrace  the  technology.   These 
factors  underscore  the  importance  of  analyzing  the  student  population  before  technology  is 
implemented. 

Table  2  shows  that  there  are  five  modules  in  this  category.   If  VTE  is  used  for  these 
modules,  a  two-way  video  and  two-way  audio  system  is  strongly  recommended. 

High-risk  candidates  are  classified  as  modules  that  focus  75  percent  or  more  on  highly 
interactive  learning  techniques  and/or  the  Exercise  Judgment  and  Change  Habits  of  Mind 
learning  outcomes.   The  lack  of  research  and  experience  in  using  VTE  to  achieve  such 
complex  learning  outcomes  suggests  that  a  large-scale  VTE  investment  for  such  learning  would 
not  only  be  a  serious  waste  of  resources  at  this  time,  but  could  also  undermine  the  learning  that 
BUMED  executives  require  to  be  skilled  MTF  Commanding  Officers  and  Executive  Officers. 
Therefore,  our  recommendation  is  to  limit  technology-based  teaching  of  such  outcomes  to 
experimental  settings.   Otherwise,  traditional,  face-to-face  instruction  should  be  used  to  ensure 
learning  effectiveness.   Table  2  shows  that  13  of  the  BUMED  modules  are  in  this  category. 
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CONCLUSIONS  AND  RECOMMENDATIONS 

The  underlying  theme  of  this  analysis  is  that  decision  makers  have  paid  insufficient 
attention  to  student  needs  and  expectations  and  educational  effectiveness  when  the 
determination  is  made  to  implement  instructional  media  such  as  VTE.   While  cost  savings  of 
technology-based  education  are  understandably  attractive,  inappropriate  technology-education 
pairings  can  undermine  learning  effectiveness,  student  and  instructor  satisfaction,  and  program 
credibility. 

This  research  demonstrates  an  approach  for  systematically  selecting  instructional  media 
for  specific  learning  applications  that  places  priority  on  the  desired  learning  outcome  and  the 
media  required  to  support  the  instructional  techniques  to  attain  that  outcome.   While  the 
research  community  has  long  discussed  the  value  of  designing  instructional  events  to  support 
learning  outcomes  (beginning  with  Gagne,  1970),  the  present  researchers  observe 
administrators'  instructional  media  choice  decisions  tend  to  be  cost  or  technology  driven  (i.e., 
the  least  costly  option  is  selected  or  the  lure  or  availability  of  a  new  technology  strongly 
influence  decision  making).   The  underlying  assumption  behind  most  of  these  decisions  is  that 
training  or  education  can  be  "delivered"  or  "transferred"  to  students  regardless  of  their 
expectations  or  the  pedagogical  needs  for  interactivity. 

These  issues  are  further  complicated  by  the  fact  that  much  of  the  research  literature 
deals  with  training.   Certainly,  the  Navy —the  audience  for  this  research—deals  extensively  in 
training  its  people  through  a  vast  and  effective  training  infrastructure.   Education,  particularly 
graduate  and  management  development  education,  is  significantly  different  from  training  with 
respect  to  learning  outcomes  and  pedagogy. 

While  we  have  encountered  numerous  institutions  using  technology-based  education, 
few  have  had  the  chance  to  conduct  research  related  to  their  efforts.  Technology  has  developed 
so  rapidly  that  many  institutions  perceive  an  urgency  to  jump  in  and  use  it  for  fear  of  falling 
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behind.   As  we  heard  from  one  institution  engaged  in  a  Master's  degree  program  via  VTE, 
"We  know  we  should  be  doing  the  research,  but  we  just  don't  have  the  time!" 

This  research  makes  an  initial  step  toward  defining  and  analyzing  variables  that  must 
be  assessed  to  ensure  that  the  balance  between  cost  and  learning  effectiveness  is  achieved  when 
media  selection  decisions  are  made.   Because  this  is  an  initial  effort,  we  recommend  the 
following  additional  research. 

1 .  Conduct  further  research  into  learning  outcomes  for  graduate  education 
and  continuing  education/management  development  programs. 

2.  Examine  the  expectations  of  adult  learners  for  technology -based,  graduate- 
level  education  and  determine  if  perceptions  change  with  exposure  to  the  various 
media.   This  type  of  work  is  particularly  important  for  the  senior  medical 
personnel  who  represent  the  target  population  for  this  study.  These  highly 
sophisticated  learners  will  not  tolerate  low  technological  fidelity  and  decreased 
opportunities  for  interaction.   As  noted  earlier,  they  feel  that  instructor- 
facilitated  discussions  among  themselves  are  catalysts  of  learning  that  are  as 
important  as  the  planned  curriculum. 

3.  Conduct  empirical  tests  to  determine  the  effects  that  various  VTE  system 
configurations  (one-way  versus  two-way  video  and  two-way  audio),  the  number 
of  sites,  the  number  of  students  at  each  site,  and  different  learning  outcomes 
have  on  learning  effectiveness.   Sound  measures  of  effectiveness  that  go  beyond 
"smile  scales"  ("Did  you  like  learning  by  VTE?")  should  be  used.   In  this 
manner,  decisions  can  be  made  to  isolate  the  types  of  training  and  education  that 
can  appropriately  be  conducted  for  many  students  at  multiple  sites  on  low-cost 
VTE  system  configurations.   Concurrently,  those  situations  in  which  such 
cofigurations  cause  a  decrement  in  learning  effectiveness  will  also  be  identified. 
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APPENDIX  A 


Module  1: 

Module  2: 

Module  3: 

Module  4: 

Module  5: 

Module  6: 

Module  7: 

Module  8: 

Module  9: 

Module  11 

Module  12 

Module  13 

Module  15 

Module  16 

Module  17 

Module  18 

Module  22 

Module  24 

Module  25 

Module  26 

Module  30 

Module  31 

Module  32 

Module  33 

Module  34 

Module  35 

Understanding  and  Effectively  Using  Communication  Media  (4  hours) 

Communicating  Vision  and  Change  in  a  Complex  System  (4  hours) 

Empowering  Associates  Through  Strategic  Feedback  (4  hours) 

Bottom  Line  and  High  Impact  Health  Care  Communications  (8  hours) 

Listening  for  Success  (4  hours) 

Decision  Meeting  Management  (4  hours) 

A  Practical  Approach  to  Strategic  Planning  (20  hours) 

Health  Care  Marketplace  Dynamics  (16  hours) 

Stakeholder  Management  and  Mapping  the  Health  Care  Environment  (8  hours) 

Total  Quality  Leadership  -  The  Theory  (8  hours) 

Total  Quality  Leadership  -  The  Context  (8  hours) 

Total  Quality  Leadership  -  The  Application  (8  hours) 

Tools  for  Effective  Management  Under  Risk  and  Uncertainty  (8  hours) 

Analytic  Tools  to  Support  Decision  Making  in  Health  Care  (8  hours) 

Cost-Output  Analysis  -  A  Decision  Making  Support  Tool  (8  hours) 

Effective  Information  Systems  (4  hours) 

The  Evolution  of  Health  Care  Market  Dynamics  (8  hours) 

Budgeting  for  Defense  Health  (4  hours) 

Military  Medical  Acquisition  Management  (4  hours) 

Ethical  Conduct  Requirements  in  Acquisition  Management  (4  hours) 

The  Accounting  Model  and  Financial  Reporting  (8  hours) 

Management  Accounting  -  Application  to  Health  Care  (8  hours) 

Considerations  in  Health  Care  -  Make  or  Buy  Decisions  (8  hours) 

Regulatory  Aspects  of  Managing  Civilian  Personnel  (8  hours) 

Materials  Management  (4  hours) 

Health  Care  Facilities  Management  (4  hours) 
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Module  36 


Module  37 


Module  38 


Module  40 


Module  41 


Module  42 


Module  45 


Module  46 


Module  47 


Module  48 


Module  49 
Module  50 


Module  51 


Productivity  Measurement  and  Incentives  (10  hours) 

Understanding  Non-Financial  Resource  Allocation  (4  hours) 

Defense  Acquisition  Process  and  Health  Care  Implications  (8  hours) 

Transition  Management  -  Executive  Considerations  in  Implementing  Strategic 

Plans  (16  hours) 

Managing  Change  in  a  Military  Healthcare  Setting  (12  hours) 

Managing  Change  in  Healthcare  -  Creating  a  Learning  Organization  (16  hours) 

Leadership  and  Motivation  (8  hours) 

Managing  Teams  in  Complex  Health  Care  Environments  (8  hours) 

Working  in  Teams  (8  hours) 

Conflict  Management  and  Negotiation  (4  hours) 

Empowerment  in  Navy  Health  Care  (4  hours) 

Strategic  Management  (4  hours) 

Medical  Readiness  (4  hours) 
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APPENDIX  B 

September  5,  1996 

From:       Alice  Crawford  and  Jim  Suchan 

To: 

Subj:        EME  Module  # 

In  response  to  a  BUMED  committment  to  evaluate  how  well  various  instructional  media 
apply  to  EME  module  delivery,  we  need  to  collect  some  information  from  you  about  your 
module(s).   Please  answer  the  questions  listed  below  and  return  this  memo  to  Alice  by 
September  14th. 

We  are  asking  you  to: 

*  categorize  each  of  your  modules  by  its  learning  outcome(s), 

*  determine  the  pedagogical  techniques  you  use  to  achieve  those  outcomes,  and 

*  estimate  your  module  content's  shelf  life,  in  other  words,  the  extent  to  which  you 
believe  your  module  content  will  change  within  the  next  three  years. 

This  information  will  help  us  determine  how  well  suited  each  module  is  for  traditional 
classroom  education,  video  teleconferencing,  or  a  new  form  of  computer-based  instruction 
called  digitized  versatile  disc.   Please  answer  the  following  questions  about  your  modules. 

1 .         Read  carefully  the  description  of  each  of  the  following  learning  outcomes. 

Classify  your  module  by  the  outcomes  that  best  describe  what  you  are  trying  to 
accomplish  in  that  module. 
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You  may  choose  one  or  more  of  the  four  outcomes  when  classifying  your 
module.   Assign  a  percentage  to  each  outcome.   Your  total  for  all  four 
outcomes  should  add  up  to  100  percent. 

a.  Know  and  Supply  Information.  The  outcome  is  student  understanding 
of  concepts,  rules,  regulations,  principles,  and/or  procedures.   For 
example,  the  Director  for  Administration  of  an  MTF  could  inform  the 
CO  about  financial  guarantees  that  are  part  of  contracts  let  to  civilian 
health-care  providers.  The  success  of  the  learning  experience,  if  a  test 
were  given,  is  measured  by  how  well  students  remember  information. 

Percent  = 

b.  Apply  Information  Within  Structured  Situations.  The  outcome  is  to 
teach  executives  to  apply  knowledge  to  structured  problem  solving 
situations.   For  example,  the  executive  could  apply  the  correct  contract 
regulation  to  a  procurement  request.   Decisions  and  solutions  to 
problems  may  be  judged  as  correct  or  incorrect  since  there  is  only  one 
right  answer,  correct  process,  or  correct  procedure  to  follow. 

Percent  = 

c.       Exercise  Judgment  in  the  Face  of  Uncertainty.  The  outcome  is  to 

teach  the  executive  to  analyze  a  multitude  of  variables  to  make  decisions 
and  solve  problems  in  the  absence  of  firm  guidelines  or  rules.   For 
example,  the  executive  might  analyze  the  organizational  issues  that 
impact  the  decision  to  change  reward  systems  instead  of  organizational 
structure  to  enhance  motivation;  there  may  not  be  one  "right"  decision. 
Measurement  of  this  outcome  is  subjective  because  the  executive  must 
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learn  to  determine  which  decisions  and  solutions  have  the  highest 
probability  of  success. 

Percent  = 

d.       Understand  and  Change  Habits  of  Mind.  This  outcome  results  in 
executives  becoming  more  self-conscious  of  the  strategies  they 
customarily  use  to  assess  complex  situations  and  more  aware  of  the 
assumptions  that  are  the  foundation  of  those  strategies.  An  example  of 
this  outcome  would  be  executives  recognizing  they  think  of 
communication  as  information  transfer,  understanding  how  this  view 
limits  their  communication  behavior,  and  uncovering  a  different  set  of 
assumptions  (e.g.,  communication  as  shared  understanding)  that  would 
reveal  fundamentally  different  approaches  to  communication  issues.   The 
outcome  may  not  immediately  create  a  better  repertoire  of  strategies,  but 
it  will  increase  awareness  of  patterns  of  thinking,  the  limitations  of  those 
patterns,  and  how  they  influence  behavior. 

Percent  = 
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2.         Please  review  the  instructional  techniques  listed  below.   Assign  percentages  that 
reflect  the  extent  to  which  you  use  that  technique  when  delivering  your  module. 

Percent 

Lecture  

Question/Answer  

Discussion  

Case  Analysis  

Group  Exercises  

Role  Plays  

simulations  

Other  (Specify) 


Total  =    100% 

3.         To  what  extent  do  you  believe  your  module  content  will  change  in  the  next  three 
years?  Circle  the  percent  range  that  best  applies. 

0-20%        41-60%        61-80%        81  >% 

Thank  you  for  your  help.   Please  add  any  comments  you  may  have. 
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